Sparsistency of 1 -regularized M -estimators: Supplementary Material 1 Auxiliary Result for the Non-structured Case

The proof is based on the optimality conditions on β̂ for the original problem, and those on β̌ for the restricted problem. We first observe that β̌n exists, since the function x 7→ ‖x‖1 is coercive. Recall that β̌n is assumed to be uniquely defined. To achieve sparsistency, it suffices that β̂n = β̌n and supp β̌n = suppβ ∗. We derive sufficient conditions for β̂n = β̌n in Lemma 2.1, and make this sufficient condition explicitly dependent on the problem parameters in Lemma 2.2. This lemma will require that ∥∥β̌n − β∗∥∥2 ≤ Rn for some Rn > 0. We will derive an estimation error bound of the form ∥∥β̌n − β∗∥∥2 ≤ rn in Lemma 2.4. We will then conclude that β̂n = β̌n if rn ≤ Rn and the assumptions in Lemma 2.2 are satisfied, from which it will follow that sign β̌ = signβ∗ provided that βmin ≥ rn.